Big data is a dataset including massive data. The data may be referred to as subdata of the big data. Only a small amount of subdata of the big data has a relatively high value for users. To help users browse big data, the big data can be processed currently, and subdata that is included in the big data and that has a relatively high value is provided to the users preferentially. For example, in the search engine field, a search engine obtains, by means of search, numerous search results according to a keyword entered by a user. The numerous search results form a piece of big data. The search engine processes the big data to obtain a search result that has a relatively high value for the user, and provides the search result to the user preferentially.
Currently, big data can be processed in the following process A mapping module performs map processing on to-be-processed big data and outputs at least one key-value pair corresponding to subdata of the big data. A key in the key-value pair corresponding to the subdata is the subdata, and a value is another piece of subdata, having a preset relationship with the key, of the big data. Then, all key-value pairs including a same key are assigned to a reduce processing module in a reduce processing module set. The reduce processing module processes values in the key-value pairs and outputs a value degree of the key. Because the key is subdata, the value degree of the subdata is obtained. According to the foregoing method, a value degree of each piece of subdata of big data can be obtained, subdata with a higher value degree is more valuable for a user, and subdata included in the big data is displayed to a user according to a value degree.
In a process of implementing the present disclosure, the inventor finds the following problem.
Currently, all key-value pairs are assigned to corresponding reduce processing modules according to keys of the key-value pairs. This may result in load imbalance of all reduce processing modules.